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Abstract 

This study examined how first-year law students used an open-ended interactive videodisc 
to develop a civil rights case. The Litigation Stmt^es\ideod\sc uses a visual and textual database 
to complement traditional curriculum by simulating the experience of a junior partner in a law fimi. 
Using a set of desktop tools {cg.y a telephone to contact the client and 28 witnesses involved in the 
case; an intercom to cramiunicate with the senior partner, investigative assistant, and legal assistant; 
and a file system to review and revise documents relating to the case), students gain practical 
e3q)erience in establishing an attorney/client relationship, interviewing witnesses, evahiating 
evidence, and interpreting legal documents. 

An implementation fomiative evaluatira examined the use of two types of orienting 
instmction provided students usiqg Litigation Stmtegies:: uqguided (i.e., general instrucdons and a 
statement of the taslc), and guided (i.e., instructicms, task statement, and a list of case development 
strategies). Twenty pairs of second-semester Harvard Law students, all from the same Legal 
Methods course, volunteered to participate in the shidy. Student pairs were randomly assigned to 
guided or unguided orienting instruction to develop the case. After 90 minutes, they specified three 
^ criteria for a legal complaint: plaintiffs), defendants(s), and cause(s) of action. 

Data collection involved: a) direct observation, b) online tracking of usage patterns, c) post- 
use completion of the legal complaint, d) an individual post-use questionnaire, and e) a paired post- 
use stmctured interview. Data analysis addressed: a) shidents* pathways through the case, b) the 
relationship between mode of use and perfomance on the complaint, and c) the relationship 
between mode of use and student perceptions of their e>q)erience. 

Results included: 

1) Guided and unguided student pairs used a variety of effective case development strategies, with 
unguided pairs intuiting many of the suggestions developed for the guided orienting instmction. 

2) Guided student pairs were more likely to produce superior legal complaints than unguided pairs, 
and guided pairs with no prior ^egal experience before law school were more likely to produce 
superior complaints than unguided inexperienced pairs. 

3) Additional orienting instmction was desired by 70% of unguided pairs as well as by 40% of 
guided pairs in this study; 10% of guided pairs desired less orienting instmction. 

The goal of^is shidy was to further our knowledge of the use of videodisc simulations for 
learning by documenting two modes of use, and by documenting die resulting student performance 
and self-perceptions. These results underscore the importance of learning more about developing 
orienting instruction in cognitively-demanding environments. 
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INTRODUCTION 

In the past decade, educational researchers have studied the learning benefits of computer 
technology, expecting to provide evidence of advantages over traditional instruction. There currently 
exists little proof that computers influence student achievement more than previous educational 
innovations (Becker, 1987; Clark, 1985; Roblyer, 1985), especially when they are used to replace 
traditional methods of teaching. There exists considerable proof of the benefit of computer-based 
instruction designed to simulate experience beyond textbook and lecture learning (Bok, 1985; 
Roblyer, Castine, & King, 1988; Zollman, Noble, & Curtin, 1987). For example, computer-based 
simulations allow medical students to develop diagnostic skills before meeting a real patient; 
simulations allow engineering students to conduct stress-testing on computerized model bridges 
before designing the structural plans for an actual bridge; they can enable future NASA scientists to 
employ rocket-launching tedmiques. Thus, the computer environment provides simulated career 
experiences that enhance the traditional classroom environment. 

Law schools are beginning to use computer-based technology to teach process-oriented skills 
such as establishing client-attorney relationships, interviewir^ witnesses, and evaluating evidence. 
These skills are difficult to learn merely through reading cases or listening to another's experience. 
Based on these needs, a complex interactive videodisc program called Litigation Strategies was 
created to provide practical case development e)q>erience within the legal education curriculum. 
Because the experience is cognitively-demanding, orienting instructions were developed to bridge 
students' prior classroom knowledge and their simulated case development experience. This study 
was designed to examine the use of the program by first year Harvard Law School students. ^ 

Design of the Videodisc 

Litigatim Sttategies simulates the experience of a junior paitner in a law firm who is assigned 
to investigate an actual civil rights case. The case is introduced by a referral memo from the senior 

^ Ihis study was cmiducted v/tilt the author was a doctoral candidate at the Harvard University Graduate School of 
Education. Funding fot the study was provided by the Harvard University Assessment Seminar, with support from the 
American Video Institute, Harvard Law School^ and Lawyers' Cooperative Pt^. 
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partner in the fim. The videodisc shows a view of a lawyer's desktop. On the desk are tools 
students can use to prepare the case: a legal pad (instnictions and calculation of costs), a set of file 
folders (documents relating to the case), and an address file and telephone (communication with 
witnesses). An intercom connects students with the senior partner, an investigative assistant, and a 
legal assistant, each of whom can assist in preparing the case. Using these tools, students have 
access to a visual and textual database of information about the case that can be used in any order. 
After meeting the client, interviewing any of 28 other people involved with the case, examining 
evidence, and conducting legal research, the junior partner files a complaint in court, specifying such 
criteria as plaintiff, defendant, and causes of action. 

As the case is prepared, the program monitors students' strategies and tallies the resulting 
costs to the firm. To increase the realism, students face various tactical and ethical decisions (e.g., 
unannounced visits from the town attorney who proposes an out-of-court settlement, and from a 
newspaper reporter who wants eariy information about the case). As in real life, actions have direct 
consequences, and failure is a real possibility. However, the disc is designed to promote learning by 
experience with a hypothetical case, versus learning at the expense of a real client. Upon filing the 
complaint, students receive a printed assessment of the methods they have used in preparing the case. 
This report evaluates student performance using current legal standards. 

The development oTUtigatioa Stmt^es was a collaboration among several groups of people 
with expertise in legal education and technology evaluation: 

• Design and develq)ment by the American Video Institute at the Rochester Institute of Technology, 
the Harvard Law School Interactive Video Proiect. and Lawyers' Coor)erative Press 

• Content assistance by second and third-year students at Harvard Law School 

• Evaluation participation by first-year students at Harvard Law School 

• Evaluation support by the Harvard University Assessment Seminar a group supporting projects 
designed to study issues in teaching and learning (with additional evaluation funding from the 
American Video Institute, Harvard Law School, and Lawyers* Cooperative Press) 



Because the instructional design of Litigation Stmt^esuses an open-ended discoveiy 
approadi to developing a case, the development team wanted to learn more about how students 
actually use the videodisc. Based on an r**'1ter study with practicing lawyers and law students at 
varying levels of legal education,^ this study was designed to investigate the use of orienting 
instruction to link students' past theoretical knowledge with what they would be expected to do in this 
cognitively-demanding simulated case development environment lliis general question guided the 
research: 

How does orienting instruction affect students' use of the program and their 
performance on a post-use complaint? 

Litigation Sttatcgiesym designed to bridge the theory students learn in law school classrooms 

and the practice they will enter as l^gal professionals. As the product was being created, the goal of 

the development team was to build a better bridge by connecting students' previous learning with their 

use of the videodisc. Three general questions framed an implementation research method to address 

the experimental, open-ended design of Litigation Stwt^es. These hypotheses were tested: 

HI: Students using the guided orienting instmction will choose more effective strat^es for case 
development than studeacs using the unguided orienting instmction. 

H2: Students using the guided orienting instruction will specify more complete complaint criteria 
than students using the unguided orienting instmction. 

H3: Student perception of the videodisc environment (e.g., appeal, utility, comfort, control) will 
vary with respect to the mode of orienting instmction. 

The next section describes the methodology used to test the general hypothesis that orienting 
instruction would affect students' use of - and subsequent performance within - the complex 
computer-based environment. 



^ This research wis preceded by a pilot study during the devdopment of Litigation Stntegiei The first study collected 
data on the use of the videodisc by experienced attorneys and 2nd and 3rd year Harvard Law Sdiool students. These data 
were used 10 improve the design of the videodisc prior to the study rep(Mrted here. 
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METHOD 



The instructional design of an educaticHial product determines in part the research methods 
developed to assess its use by learners. Litigation Stmt^es addresses goals beyond the usual scope 
of the first-year law school curriculum. Thus, there is no traditional instruction to which this 
enyircMunent can be compared. To examine students' use of this case development environment under 
specific orienting conditims, an implementation formative cvahiation was conducted. 

This observational study was conducted over a 3-week period in the ^ring of 1989 at 
Harvard Law School, examining the use of two types of orienting instructions - guided and 
yngujj^. Twenty pairs of first year students were randomly placed in either the guided or unguided 
group, resulting in 10 student pairs in each group. Before beginning the legal case, all student pairs 
received 3 identical documents: 1) a case referral memo, 2) general instructions about using the 
technology, and 3) a statement of the task following the investigation (i.e. listing specific information 
on a complaint to be filed in court). In addition, guided pairs received an orienting checklist that was 
expected to help students focus on particular parts of the disc. After using the disc for 90 minutes, 
each pair specified the plaintifi(s), defendant(s), and cause(s) of action and listed anticipated prx)blems 
in bringing the case to trial. Participants then independently completed a 10-item questionnaire and 
responded to a brief structured interview with their partner. The entire experience lasted between 2 
1/2 and 3 hours. 

Measures 

In this study, multiple measures collected diverse types of data, permitting greater 
understanding of learners' experience. Two types ofmeasures were uses: 1) measures of student 
performance, and 2) measures of student perception . Student performance data were collected with: 
(a) an online computer program that tracks student pathways, (b) direct observation, and (c) a paper 
and pencil test (i.e. the complaint form). Student perception data were collected with: (a) a closed 
and open-response individual questionnaire, and (b) a brief, structured pair interview. Measures in 
each category will be briefly described here. 
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1. Measures of Student PerTormance 

fathw^Y TffCking Program- A data collection device within the Litigation Stmtegiesptogram 
monitored each use of the disc. This tracking device recorded each pathway (i.c . the series of steps 
and time elapsed between those steps) used to investigate the ca5t. This pathway was saved and later 
printed, providing a record of the sequence and duration of choices made during the investigation. 
For example, a pathway mig^t show that a student pair began their investigation by making these 
choices: 

• Conducted the initial client interview with Donald Boyd ( 1 3 minutes); 

• Ordered from the Investigative Assistant the 9 1 1 report, 5 witness statements, and a street 
diagram of the area where the incident took place ( 1 minute, 1 5 seconds); 

• Read the 91 1 report, examined the street di^^ram (7 minutes, 20 seconds); 

• Interviewed 2 eyewitnesses to the alleged incident (27 minutes); and 

• Ordered 3 legal memos fnmi the Legal Assistant (2 minutes). 

This tracking system was useful not only in collecting sequence and duration data but in 
quickly determining frequency counts within and across given data sets. Many of the advantages of 
an online data collection device are obvious. This tracking system provides a structured, easy-to- 
understand report of the sequence and duration of choices each student pair used to investigate the 
case. It added rigor and ease to the tedious process of narrating, in writing, a student's specific 
pathway through the investigation. This allowed direct observation in other areas, such as 
collaboration and conversation between partners, providing a more complete understanding of the 
thinking of pairs as they strat^gized out loud. 

Direct Observation. Data collected by the tracker program were augmented with direct 
onlooker observation (i.e., no interactim with participants after the initial introduction and assignment) 
of each student pair using the disc. Since the sequence and duration of strategies was collected by the 
tracking program, observations focused on data not collectable by machine. This included the 
presence or absence of: (a) verbal and non- verbal communication between pairs, and (b) physical 
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responses to the program (e.g., gesturing, nodding, moving toward or away from the screen). Direct 
observation of participants in this study provided several advantages: 1) the opportunity to sec and hear 
things the tracker program could not record; 2) the chance to understand, firsthand, the context in 
which the program was used by an inten led audience; and 3) the opportunity to see and hear things the 
participants may not have been willing to share in an interview or on a questionnaire. 

Paper and Pencil Complaint Form. Immediately after investigating the case, students were 
asked to specify three criteria for filing a complaint in the case: 1) the defendant(s), 2) the plaintiflls), 
and 3) the cause(s) of action. This measure was designed to replicate an actual complaint fonn used in 
court Students used a **flll in the blank^ style for completing the complaint, choosing from several 
names of people and possible causes of action in the case. They also provided a short narrative 
justifying their stated cause(s) of action (as required by Rule 1 1 of the Federal Rules of Civil 
Procedure) and described possible problems they anticipated in taking the complaint to court. 

2. Measures of Student Perception 

Individual Questionnaire. Questionnaire responses provided scale and descriptive data aoout 
individual participants' perceptions of using the videodisc environment to investigate a 1^ case. 
Questions addressed students' perceptions of the adequacy of the orienting instmction, the adequacy of 
the level of omtrol ^hey e}q)erienced using the program, their understanding of the case, their level of 
confidence in their complaint criteria, and their perceptions of woridng with a partner. To determine 
accurate individual data, participants completed the questionnaire independently before the interview 
with their partner. 

Structured Pair Interview. The interview was structured to gain broader understanding of the 
eiqperience of each student pair. Interview responses provided descriptive data about the multiple 
perspectives of the students who used the disc. Questions addressed students perceptions of their 
ability to apply prior knowledge in the videodisc environment, to gain new knowledge^ and their 
assessment of the relevance of the disc use to the first-year law sdiool curriculum. Perhaps equally 
important, the structured interview helped uncover and document effects the program developers did 



not intend to pioduce. Results were used to improve the introduction and educational use of videodisc 
iechnology. 

The measures of performance and perception were designed to relate to one or more of the 
hypotheses in the study. Table 1 indicates how each measure relates to each hypothesis. 

TABLE 1 ABOUT HERE: 
RELATIONSHIP BETWEEN MEASURES AND HYPOTHESES 

Taken together, these diverse sources of information and data provided a broad picture of 
participants* use and perceptions of the Litigation Strategics program. The use of multiple measures 
to collect various kinds of infonnation allowed cross-validatton of information from each. In the next 
section of this paper, data fro^^'i the study are presented. 



ERJC 10 



RESULTS 



This study was designed to determine the impact of orienting instruction on three outcomes: 

1) law students' case developmrat strategies in a videodisc environment; 2) their performance on a 

written legal complaint; and 3) their perceptions of their videodisc case development experience. To 

review, three hypotheses guided the study: 

HI: Students using the guided orienting instruction will choose more effective strategies for 
case development than students using the unguided orienting instruction. 

H2: Students usiqg the guided orienting instruction will specify more complete complaint 
criteria than students using the unguided orienting instruction. 

H3: Student perception of the videodisc environment (e.g., appeal, utility, comfort, control) 
will vary with respect to the mode oi orienting instruction. 

Briefly, the main findings of the hypothesis testing are presented here: 

1) Student pairs assigned guided and unguided orienting instruction used a variety of effective 
strategies to develop the case. Unguided pairs, while not receiving the checklist, intuited the use of 
many of the suggestions from the checklist. However, guided pairs used a slightly higher number of 
checklist strategies than unguided pairs. Guided pairs began factual and investigation earlier, began 
legal investigation later, and handled une>q>ected visitors mih more discretion than did unguided 
pairs. Unguided pairs made more eflicient use of their assistants on the case. These behaviors were 
encouraged on the orienting instniction checklist. 

2) Student pairs receiving guided instniction were twice as likely to produce a Superior 
complaint as those receiving unguided instniction. Unguided p^ ars producing Superior complaints 
used a slightly higher average number of checklist strategies than the guided pairs producing Superior 
complaints. However, unguided pairs producing Adequate complaints averaged the least number of 
checklist strategies of any group. Student pairs (both guided and unguided) producing Superior 
complaints conducted, on average, earlier investigative (factual) research and later legal research than 
did pairs producing Adequate complaints. Thus, for both the guided and unguided groups, the use of 
effective strategies may have translated into better performance on the l^gal complaint. In addition, 



ihere exists a strong association between guided students' success on the complaint and their having 
no prior legal experience before coming to law school. Inexperienced guided pairs are three times as 
likely to produce a Superior complaint as inexperienced unguided pairs. 

3) Students in guided and unguided modes were quite positive about their videodisc case 
development e}q>erience. Both groups rated the disc as easy and enjoyable to use, and easy to 
understand. Guided students were slightly more confident of their complaint performance than 
unguided students. However, guided students felt slightly less in control of the experience, and were 
slightly less comfortable with their level of control than unguided students. The largest difference in 
perception data occurred in students* assessment of their own learning, with guided students 
perceiving a higher level of learning when comparing the disc to other methods of case development. 
Unguided students rated the adequacy of the instructions slightly higher than did guided students. 
However, 70% of unguided students and 40% of guided students would have liked more instructions 
before beginning to develop the case. 

It should be noted d;at no guided pairs were observed using the orienting instruction in true 
''checklist'' fashion (i.e., checking back regularly to see what to do next, doing it, and checking it 
off). It was usually read over at the start of the case, mentioned only occasionally as pairs worked, 
and returned to if they became "stuck." 

This section outlines two areas in more detail: the analysis p rocess used to reach these 
conclusions, and further discussion of the results. 

ANALYSIS PROCESS 

This study ustd performance and perception measures to examine students' use of the disc, 
their performance on a post-use test (i.e., the complaint), and their perceptions of the videodisc 
e)q)erience. The analysis of data from each measure is described below. 

Performance Measures 

Pathway Data, The following steps were used to analyze the guided and unguided student 
pathways through the case. Fiist, a matrix was created listing the sequential case development 
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strategies of each pair. Thisn irix included the items listed in the orienting checklist. For each mode 
of use (guided and unguided), these strategies were tallied and a mean number of checklist choices 
was detennined. Acnial checklist choices could lange from 0 to 9. A pail's checklist choice total was 
obtained by summing their actual choices that were also listed on the checklist. In this study, 
diecklist choice totals ranged from 4 to 9. (This table is illustrated in a subsequent discussion of 
hypothesis testing.) 

Obsefvation Data, Ccmveisation between partners wns examined using a method described by 
Daiute ( 1989). Because collaboration allows both muung aloud individually and deliberating with a 
partner, each speaker's '*unintemipted contribution to the conversation** was counted as one **talk 
turn** (p. 7). Using this system, transcripts of conversation were coded and counted, arriving at a 
number of talk turns for each pair. Omversation levels were then labelled **his^,** medium,** or 
**low.** An effort was made to further discriminate between **creative** and **critical** types of 
conversation (Perkins, 1984) and between musing and deliberating conversational behavior (viz, 
Daiute, 1989). Eachofthesemethods,althou^potcntially valuable to consider, was diflicult to 
apply. There were two reasons for this. First, conversation generated by the program would often 
fall into both the creative and critical conversation categories, or into both the musing and deliberating 
categories. "Second, verbal interaction in diis study occurred not only between partners but among the 
human pair and the characters on the screen.^ As such, three general levels of conversation (Hig^, 
Medium, and Low) were analyzed for purposes of this study. Conversation levels were rated 
numerically (Ifigh - 3, Medium - 2, and Low -> 1) to arrive at conversation scores for each pair. 
Mean conversation scores were then calculated for each group. 

Performance Data, Twenty complaints were prepared - 10 by guided pairs, 10 by unguided 
pairs. For analysis purposes, three legal experts med each of the twenty complaints blindly (i.e., 
without knowledge of the mode of orienting instmction relating to each complaint). Initially, each 
expert was asked to devise a three-part system. First, they developed their own criteria for what 



While the potential for leaeaich in this area is very interesting, it will not be fully-developed in this report. 
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cmstitutes an "^efTective'' or high-quality complaint. Next, they each applied these criteria to judge the 
quality of each complaint Finally, they each classified the 20 complaints according to their quality. 

Not surprisingly, the criterion devised for an **effective** complaint were quite similar among 
the legal experts. However, the ^plicaticm of the criteria resulted in considerable variation among the 
experts. For example, each expert listed (as criteria for assessing the quality of the complaints) 
students* ability to thorou^ly describe problems of proof, such as listing probable cause, adverse 
witnesses, and the possible immunity of the arresting ofiicer. Periiaps understandably, the 
application of such subjective criteria led to considerable variation in the level of importance each 
expert assigned this area. This resulted in three diverse systems for classifying complaint quality: 
one was a numerical ranking, two were categorizations. Expert 1 ranked complaints in numerical 
order, from highest to lowest Expert 2 developed 3 categories (i.e.. Adequate, Fair, and Poor) and 
distributed the 20 complaints across all categories. Expert 3 developed 5 distinct categories (i.e.. 
Excellent, Very Good, Good, Fair, and Poor) and placed complaints only in the top three. 

This initial rating and classifying of complaints was interesting and periiaps indicative of the 
way we as teachers examine and judge student work. However, because the experts each devised a 
unique method of classifying the quality of student complaints, there was great diversity in the 
categories into which complaints were placed. This did not allow a common way to examine the 
strength of relationships between variables in this study. Thus, I initiated a second round of study of 
the complaints, this time asking each expert to place the 20 complaints in 2 equal groups: the upper 
50% and the lower 50%. In most cases, this second rating required reconsidering complaints near 
''the middle of the pack," and reassigning those complaints to either the upper or lower group. This 
reconsideration of the complaints resulted in three independent ratings of the complaints into two 
categories. 

The purpose of examining the strength of the relationship between modes of orienting 
instruction and complaint performance was to learn more about constmcting efficient orienting 
instiuction for introducing this videodisc to law students. For this reason, the next step was to devise 
a way to combine the complaint ratings on one matrix in order that performance could be discussed. 




To do this, I combined the three ticpttt ratings on one 2x2 contingency table. Using this method* 
eveiy complaint was fated similariy by at least two of the three experts; many were rated similarly by 
all three experts. To distinguish between hi^er and lower complaint quality* the labels ''Superior^ 
and '^Adequate** are used in this report. 

To sunmiarize, this rating system resulted in the cat^orizing of 20 complaints prepared by 
student pairs - 10 from the guided orienting instmction mode and 10 from the unguided orienting 
instruction mode. These complaint ratings are used in the subsequent examination of Hypotheses I 
and 2. Hypothesis 3 measures the use, comprdiensibility, and appeal of the Litigation Siiategies 
program by the entire sample of 40 individuals. 

Oaestionnaire and Interview Data . There were a total of 10 items on the questionnaire. 
Responses were collected using a 5-point Likert scale. Two checklist questions and two short answer 
questions collected deta about students' prior experience, how they made decisions in the case, and 
their feeliqg about developing the case with a partner. For each question, a mean and standard 
deviation was calculated. For purposes of analysis, the alpha was set at the .05 level (one-tailed) for 
hypothesis testing. These data sets were examined using statistical tests since questionnaire data were 
numerical and included responses from 40 subjects. The t '^h' ^ of data analysis in this study 
involved non-statistical examination of qualitative data generated by the 1 0 guided and 1 0 unguided 
student pairs. 

Qualitative data were collected using the structured pair interview. Perceptions of guided and 
unguided pairs were categorized according to several topics: new learning; least appealing aspects of 
the videodisc experience; most appealing aspects of the videodisc experience; suggestions for 
improving the disc; adeq;;acy of instructions; and recommendations regarding adding the Litigatim 
Stiatcgiestxpetieact to the first year law school curriculum. The next part of thii> section reports 
results for each of the three hypotheses guiding the study. 
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HYPOTHESIS TFSTING 



HYPOTHESIS 1: Effect of Orienting Itistnictions on U.v> of thi. Virli^i.^ 

Hypothesis 1 predicts that guided pa'.rs will choose more effective case devdopment strategies 
than unguided pairs. Two areas will be reported in this anal>'sis: 1) pathways through the case, and 
2) conversation topics that emerged as guided and unguided student pairs prepared the case. 

1. Pathways 

Use of the checklist was e?q>ected to lead to the efficient prej^ticm of an effective complaint. 
Twenty pathways were examined: 10 from guided paiis, 10 from unguided pairs. Table 2 presents 
the strategies used by each group within guided and unguided modes,^ as compared with the checklist 
suggestions. As indicated iu the table, student pairs used diverse strategies in various sequences (i.e., 
T - the first strategy used, ""T - the second strategy, and so on). The category Total Choices** 
indicates the number of checklist choices each pair used, with a possible total of 9. Guided pairs in 
the study used a slighdy hi^er average number of checklist strategies (6.7 per pair) than unguided 
pairs (5.8 per pair). Thus, ttxre was only a slight difference between the guided and unguided pairs 
on strategy choices, despite the availability of the checklist by guided pairs. 



TABLE 2 ABOUT HERE: 
CHECKLIST CHOICES 



Also indicated in Table 2 are two additional categories: Efficiency and Discretion. Under the 
category "^flicienc/' are plus and minus symbols. Because the checklist included suggestions 
regarding efficient work habits, this category is reported here. A ^{^Y efficiency rating indicates that 
the pair used their Investigative Assistant (lA) and Legal Assistant (LA) efficiently. That is, they 

^ It should be noted that although unguided pairs were not provided the orienting checklist, the list of checklist 
strategies were used in this analysis as a basis for determining whether the use of the checklist correlates with the 
aeatim of a superior legal complaint by student pairs. 
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devised tasks for the assistant » and then worked on another aspect of the case, versus merely waiting 
for the assistant to complete a task. A ''(-f rating in this cat^oiy indicates inefficient use of an 
assistant. Of the 32 uses of assistants by guided pairs in this study, 69% were efficient. Of 29 uses 
of assistants by unguided pairs, 80% were efllcient. This outcome leads to the question: What in the 
checklist might lead guided pairs to work less efliciently than unguided pairs? Is it eflicient to work 
on another part of the program while an assistant completes an assigned task? These questions will be 
examined in the discussion of results. 

The checklist provided guided pairs also suggested using discretion with unexpected visitors. 
The final category on Table 2, ''Discretion,*' reports ratings of 62% for guided student pairs, and 45% 
for unguided student pairs. Positive or negative feedback within the program was provided to 
students following interactions with these visitors, according to the way they handled the interaction. 
Positive (+) discretion ratings were awarded student pairs receiving positive feedback following at 
least 75% of these interactions. 

Despite the finding that overall the mean number of checklist choices differed only slighdy 
between groups, there were interestuig variations in incfividual strategies between groups. For 
example, guided pairs were aware of the existence of specific information about the case (i.e., a map 
of the area in and around Land's End Road, where the incident occurred, and a series of character 
statements describing potential witnesses). Few unguided shidents explored this information. In 
addition, there are differences between guided and unguided pairs in the mean time at which they 
initiated both investigative and legal research. These results will be discussed relative to Hypothesis 
2. 

A second area in which guided and unguided pairs varied involved their level of conversation 
and topics discussed as they developed the legal case. 
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2. C onvcnation 

It was expected that omveisation between partners might influence their general case 
development strategics and complaint performance. While it is difiicult to point to specific outcomes 
of conversation between partners, Tables 3 and 4 illustrate some trends that emerged. 

Table 3 illustrates complaint ratings, general conversation levels (High, Medium, and Lowi 
based on talk turns, and major topics of conversation. Guided pairs rarely discussed the checklist, lu 
fact, only 1 pair actually mentioned it by name after reading if initially, and only a few pairs referred 
back to it as they worked. However, between steps taken by guided pairs, there was frequent 
discussion of checklist options (although not referred to as such), indicating that many guided pairs 
had mentally stored the suggestions.^ 

TABLE 3 ABOUT HERE: 
COMPLAINT RATING, LEVEL AND TOPICS OF CONVERSATION 

Major topics of conversation included issues ranging from the client's stoiy to the checklist. 
As indicated in Table 3, greatest variation between guided and unguided groups involved: other 
witnesses, reaction to the program, connectedness of the videodisc program to what students do in 
class, and actual procedures for using tlx program. Guided pairs more often discussed other 
witnesses, their reaction to the program, and the classroom connection; unguided pairs focused more 
on procedures for using the program. 

Table 4 reports a mean conversation conversation score for guided and unguided groups. 
Conversation scores were determined using a point system, where High - 3, Medium - 2, and Low 
- 1. Overall, the mean conversation score of the group of 10 guided pairs was sli^tly higher (2.1) 
than that of the group of 10 unguided pairs ( 1 .7). 



^ Although this study did not focus directly on the ability of learners to devdop and retain mental frameworks for 
working in this enviitmmentt this is an area in which future research would be informative. 
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TABLE 4 ABOUT HERE: 
MEAN CONVERSATION SCORES 



Several trends emeiged in conversation between partners, involving: interaction; use of 
assistants; considering alternatives; limiting premature decisions about the case; and pooling individual 
knowledge. Each area is addressed below» in a selection of various conversations across both guided 
and unguided groups. 

Interaction . As mentioned eariier, conversation occurred not only between partners but 
between iuid among each partner and various characters on the screen. For example, consider this 
conversation between a student pair and the character onscreen after successfully completing the initial 
retainer arrangement with their client, Donald Boyd: 

SI: Okay, so here he is in our ofRce. Do we want bin whole stoiy now, or do we put him ofl? 

S2: He's here, let's go for it. See what he has to say about what happened that day. 

(They spend 15 minutes getting what they assume is Boyd's complete story.) 

S2: Okay. We should check into his background before we let him go, don't you think? 

SI: Yeah, but do it gently. We don't want to get him pissed ofT. 

(They choose the 'Personal' section from the 'Background' menu.) 

S2: Race? Ask him what race he is? 

SI: What do you mean? He's Black! 

S2: But I just want to see what he says. 

SI: Well... 

(S2 chooses 'What is your race?' before they both agree to do this.) 

Boyd: What do you mean, what race am I? Use your eyes - get serious! 

SI (to S2, angrily): Fm not listening to you any more. This guy's getting mad. Give me the 
mouse! 

SI (to Boyd): Mr. Boyd, take it easy. Fm in control now. We're sorry. 

ERsLC 1 n 16 



Later, the same pair asks Boyd if he has anything else to say. They realize that Boyd 
neglected to share some critical parts of his story. The second student is still in control of the mouse. 
S2: Okay Mr. Boyd, we're just about done with you. Anything else to add? 
(Boyd tells them he has been arrested previously.) 
S2 (to Boyd): Why you little ... 

SI (to S2): Easy, now, easy. (To Boyd on the screen): Mr. Boyd, we could help you the most 
if you would tell us the whole story please. Now what else happened? 

S2(toSl): Is this guy going to level with us or what? 

SI: Look, this is probably just like how it happens in real life. You have to ^et them talking and 
talking and then they Anally tell it all ... (laughter) ... and that's why it costs them so much to 
have a lawyer. 

These examples illustrate interaction that occurred reguhriy between and among the students and the 

characters in the videodisc pipgram. 

Use of Assistants , As stated earlier, guided pairs were more discrete (62% positive ratiiig) 

than unguided pairs (45% positive rating) when discussing the case with une>q>ected visitors. 

However, encounters with the town attorney and a newspaper reporter, even when pairs took 

questionable action, seemed to resuh in enjoyable learning. For instance, one pair agreed to see the 

reporter, and then realized that they were being asked to answer questions regarding the case to which 

they peihaps should not respond: 

SI: Gee, this guy wants to know whether there will be criminal charges filed. What do we 
say? 

S2: Well, our only choices are yes or no. There's no 'maybe.' 
SI: So, I guess' well have to say yes? 
S2: Yeah, see what happens if we do. 

(This results in a sternly-worded response from the senior partner, reminding them not to give 
interviews to reporters at this p^^Lit.) 

SI: Oh-oh, a note from the boss. 

S2: He didn't like that. He didn't like that slM- 

SI: So we were over-eager. Well remember not to invite him in next time. 
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Considering Altemat^'y^^ Working with a partner seemed to lead to regular examination of 

many possible strategies before actually taking action. These exchanges resulted in making decisions 

that were informed by two peoples* thinking. In the example below, a pair is discussing the 

motivation of Carol Weller in phoning in the 9 1 1 report. 

SI: Well, Carol did phone in the report because she said she saw the junker car parked in front 
of the house, right? 

S2: Yeah. Ijust wish there were some way we could get a look at the house. 
S 1 : So let's see if our Investigative Assistant can help. 
S2: Huh? 

S 1 : This lA has evidence about the case. Maybe he has picmres of the house. 
S2: I forgot about that. Let's check. 

(They examine the house photos, and consider Wellei's reasoning.) 

SI: Look at that ladder-lookiqg thing going up the wall of the house. What do you call that? A 
trellis. Yes, a trellis. Could that be ...? 

S2: Thaf s it! Thaf s it! Thafs what she saw and she called in to report a ladder a gainst the 
house! How are we going to deal with this? 

S 1: Well, we could go right to the police and tell them we have these photos. 

S2: Or we could just stash the photos and get the legal memoranda we need for court. 

31: Or we could go have lunch, now that weVe cracked the case (laughing). 

Limiting Premature Decisions About the Case. Pairs often challenged each othei's early 
theories about the case, increasing the range of options they discussed concerning people's roles in 
the case. 

S 1: Well, here's what I think. I think we can get them [police] on assault and battery, just by 
the way they threw him out of the car! 

S2: Threw him out of the car? Lefs go check that. I thought it sounded not quite that bad. 
(In checking this, they realize only one ofTicer was involved.) 

SI: So he didn't get quite so roughed up as I thought. What other causes of action can you 
think of that we can back up? 

S2: Lefs read about possible causes in the memos, okay? Maybe something about his civil 
rights being violated. 
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Pooling Individual Knowledge. The phrase "two heads are better than one" was mentioned 8 
times in lesponse to an interview question about the value of working with a partner. Student pairs 
were overwhelmingly positive about working on the case with a partner. Two reasons frequendy 
cited were the chance to pool their legal knowledge, and to niake more informed decisions than they 
would individually. In one case, a pair was worlung with the wording of a legal memo. 
SI : So what does it mean, to 'cleariy establish' something? 

S2: Fve read about this in class. It means that under the law, ifs the cop's job to know about 
other cases like the one he's involved in. 

SI: Oh, like 'past precedent?' 

S2: Exactly. 

SI: We're so smart. 

In summary, pathways for guided and unguided modes varied in several ways, including the 
mean number of checklist strategies chosen, the tuning of the first use of the investigative assistant 
and the 1^ assistant, and pairs' use of discretion with unannounced visitors. While overall pathway 
variations exist between the guided and unguided groups in this study, the actual influence of the 
orienting instniction remains unclear. 

HYPOTHESIS 2: Effects of Orienting Instnic tion on Complaint Performance 

The second hypothesis examines the influence of orienting instruction on students' ability to 
specify three criteria on a 1^ complaint The following tables present data showing the relationships 
between complaint performance and several other variables: 

1. Mode of Orienting Instmction 

2. Checklist Choices 

3. Conversation Scores 

4. Time oflnvestigative and Legal Research 

5. Prior Legal Experience 

1. Mode of Orienting Instniction 

The constmction of the legal complaint was described eariier. To analyze the way in which 
orienting instmction may influence complaint perfomiance, the raw data was transfomied as follows. 
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A matrix of all the complaints was produced, listing these data: plaintifKs), defendant(s), causc(s) of 
action, and possible problems pairs expected when bringing the case to trial. The matrix was then 
^ven to three experienced attorneys who rated and classified each complaint according to criteria each 
devised independently. 

Each legal expert developed, independently, rating criteria that was quite similar. Complaints 
should: name more than one plaintiff in the case; state specific causes of action (e.g., false 
imprisonment, unlawful arrest, assauk and battery); support causes of action under a specific state or 
federal statute (e.g., the Massachusetts Civil Rights Act); and recognize problems th&t might surface 
in court These include problems of proof (e.g., proving that race was a factor in the incident, 
proving that the officer's action was unreasonable); problems of immunity (e.g., city and state may be 
immune frran punitive damages); and problems of reputation (e.g., prior police record of the client, 
officer's lengthy service on police force). Using these rating criteria, each expert placed complaints in 
top and bo^iom gioups. The application of the rating criteria resulted in the complaint performance 
matrix in Table 5. 

TABLE 5 ABOUT HERE: 
COMPLAINT PERFORMANCE 

Table 5 shows the association between complaint ratings for guided and unguided student 
pairs. Sixty percent of pairs receiving guided orienting instruction produced Superior complaints, 
versus 40% of pairs receiving unguided orienting instruction. The next part of this section considers 
an important variable relating to performance, checklist choices. 

2. Checklist Choices 

Pairs producing Superior complaints (guided and unguided) averaged 6.8 checklist choices. 
Pairs producing Adequate complaints (guided and unguided) averaged 6 checklist choices. 
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Table 6 shows the strength of the relationship between the number of checklist items chosen 
and the quality of the complaint filed by guided and unguided student pairs. Based on these data, 
guided pairs producing Superior complaints averaged 6.5 checklist choices; ui^ided pairs producing 
Superior complaints averaged 7 checklist choices, for a combined average of 6.8 checklist choices for 
the group of Superior complaints. Also, guided pairs producing Adequate complaints averaged 7 
checklist choices; uqguided pairs producing Adequate complaints averaged 5 checklist choices, for a 
combined average of 6 checklist choices for the group of Adequate complaints. 

TABLE 6 ABOUT HERE: 
MEAN NUMBER OF CHECKLIST CHOICES USED 

3. Conversation Scores and Complaint Ratings 

Table 7 shows the mean conversation scores associated with complaints prepared by guided 
and imguided groups. Interestingly, conversation scores for guided and unguided pairs with Superior 
complaints were identical to guided and unguided pairs with Adequate complaints. However, guided 
pairs had a slightly higher overall conversation score (2. 1) than unguided pairs ( 1 .7). 

TABLE 7 ABOUT HERE: 
MEAN CONVERSATION SCORES AND COMPLAINT PERFORMANCE 

4. Time of Investigative and Legal Research 

Table 8 illustrates the relationship between complaint perfoi .nance and the time at which 
investigative and legal research was initiated. As referenced in Table 2, the ''mean step"* at which a 
checklist choice was used refers to when that step occurred in the total sequence of choices used by a 
student pair. In general, guided pairs with Superior complaints began conducting investigative 
research earlier (mean step of 2.8) than all other pair types: this includes unguided pairs with 
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Superior complaints (mean step of 3.3); unguided pairs wiA Adequate complaints (mean step of 3.3); 
and guided pairs with Adequate complaints (mean step of 4.8). In some cases, investigative research 
occurred even before the guided pairs established a relationship with their client. 

TABLE 8 ABOUT HERE: 
MEAN STEP OF miTIATION OF INVESTIGATIVE AND LEGAL RESEARCH 

Table 8 also illustrates the mean step at which legal researdi was initiated by guided and 
unguided groups. Guided pairs producing Superior and Adequate complaints conducted later legal 
research (mean steps of 8.2 and 7.6, respectively) tlian unguided pairs producing Superior and 
Adequate complaints (mean steps of 5.8 and 4.2, respectively). These flndings will be discussed 
later. 

5. Prior Lei al Experience 

Of 20 teams of students in the study, 13 contained at least one individual having some legal 
experience before beginning law school. This ranged from doing a simple series of short 
investigative memos as a two-week cleik in a law office, to the completion of 3-4 months of fulltime 
clerking experience with a law firm. Thus, it is important to examine the possible relationship 
between prior legal experience and a team's success on the legal complaint after developing the 
videodisc case. 

TABLE 9 ABOUT HERE: 
PRIOR LEGAL EXPERIENCE 

Table 9 displays the variance in complaint quality between pairs with and without prior legal 
experience. There appears to be no relationship befiveen prior experience and either guided or 
unguided pairs' success on the complaint. However, there exists an association between guided 
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instniction and Superior complaint ratings for pairs with UQ prior instniction. That is, inexperienced 
miM pairs are three times more likely to produce a Superior complaint than inexperienced unguided 
pairs. 

In summary, 3 legal experts developed a system for rating and classifying the legal 
complaints. Matrices were developed to examine possible relaticxiships between complaint quality 
and mode of orienting instruction, checklist choices, conversation scores, time of initiation of 
investigative and legal research, and the prior legal experience of student pairs. The relationship 
between orienting instruction and quality of complaint is slight, although pairs with Superior 
complaints used slightly more checklist choices and weit slightly more conversant than pairs with 
Adequate complaints. Superior ccmiplaints were associated with early investigative research and late 
legal research. Finally, there is an associaticm between Superior complaint performance and guided 
student pairs with no prior legal ejq)erience. These relationships will be further discussed in a 
subsequent section of this paper. 

HYPOTHESIS 3: Effects of Vide odisc Experience on Student Pi^icmtions 

The third hypothesis suggests that student perceptions of their videodisc experience will vary 
with req>ect to uieir assigned mode of orienting instniction. 

General Perceptions 

To evahiate this hypothesis, I examined two types of student responses: oral and written. Oral 
discussion during the case was recorded and analyzed, as well as oral responses to a post-use 
structured interview with each student pair. Written data were collected with a 1 0-item 5-point Ukert 
scale questionnaire and 4 open-ended response questions. 

The questionnaire examined the influence of the videodisc case development experience on 
students* perceptions in several areas: 

• ease of use of the technology; 

• clarity ofthe presentation of the case; 

• adequacy of instructions; 

• confidence in die complaint criteria specified; 
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• level ofcontrole]q)erienced while using the disc; 

• level ofcomfoit with perceived control; 

• upptal of woildqg with a partner to develop the case; 

• enjoyment ofthe investigation experience; 

• resemblance oftfae videodisc case to real life; and 

• level oflcaming experienced in the process. 

Among the advantages of using interactive videodisc for learning, Miller (1988) reports that 
interactive video offers a non-thieatening, non-judgmental environment that is fun and challenging. 
Participants in this study - those with guided and those with unguided orienting instruction - 
responded favorably to questions about their enjoyment of the disc. 

To analyze the questionnaire data, a mean response for each group was calculated for each 
question (see Table 10). Meaii<( and standard deviations were calculated for each question across both 
modes of orienting instmction. The deviation from the mean of eacn question was not significant. 
Although there were no statistically significant differences on any question, it should be noted that 
guided students are: 

• slightly mS2IS confident in the criteria they stipulated in their complaints ; 

• slightly 1^ comfortable with the level of control they experienced while using the program (i.e. 
freedom to use the program in the way you chose, to work toward the goals of the complaint); 

• slightly msoi positive in viewing the experience as one in which they learned about the process 
of investigating a legal case. 

Unguided students report: 

• sli^tly higher satisfaction with the written instmctions (i.e., **case referral memo, instructions 
for using the program**) they used to develop the case; and 

•slightly higher enjoymer/ of the videodisc experience. 

TABLE 10 ABOUT HERE: 
MEANS AND STANDARD DEVIATIONS FOR QUESTIONNAIRE ITEMS 

Student responses to the questionnaire and interview items also yielded information about their 
perceptions of the instnicticm they received. Table 1 1 displays the response of guided and unguided 
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individuals to the cpestion: Describe your reaction to the instruction you received prior to starting the 
case - would you have liked more, less, or was it the right amount?** 

TABLE n ABOUT HERE: 
ADEQUACY OF INSTRUCTIONS 

This table illustrates the level of satisfaction of individuals in each mode of use. In the guided 
group, 40% (8 individuals) desired more pre-instniction, versus 70% ( 14 individuals) of the 
unguided group. In the guided group, 50% ( 10 individuals) were satisfied with the instiuction they 
received, versus 30% (6 individuals) in the unguided group. Ten percent (2 individuals) of the 
guided group wanted less instmction. Anecdotal accounts from students are presented in 3 general 
categories: 1) new learning, 2) least favorite parts of the videodisc experience, and 3) favorite parts of 
the experience. 

1. New Learning 

All students were asked whether they experienced new learning as they used LiH^gation 
Stmtegicsi Guided participants (8/20) responded to 'Ms question by comparing the Litigation 
Stiategicsptoccss to the process used in their Legal Methods course. One group described the 
classroom procedure as 'thaving the case] handed to you on a platter.** In contrast. Litigation 
Stratctgieswis *1ike getting a lump cf clay and having to shape the case, using both the law and the 
facts.** Among their new realizations about litigation: 

• The importance of talking to people on the fringe of the case as well as the obvious witnesses; 

• It is diflicult to interpret people's comments, judge their various perspectives about the case, and 

then choose good avenues to pursue. 

Overall, Litigation Stmt^esvms **a way to apply classroom thinking,** and **a chance to work 
with the laws we have read about in class by first learning how to get the facts, next apply the facts to 
the law, and then get more facts.** 

Unguided students reported learning amilar types of practical skills: 
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• Efiiciency in the videodisc experience depended on first doing legal research, then 

interviewing, and then doing more legal research. 

• Real life is ''filled with deadends in interviews,** ''sometimes you need to shut people up," and 

"not all witnesses are going to be as helpful as the ones we had in class." 

• It is important to "consider all contingencies when an attorney is under time and money 

constraints." 

• A real case "consists of a breadth of sources of information like this to work with, like memos. 

articles, evidence, pleadings, and motions." 

2. What Did You Like Least? 

Guided pairs (5/10) cited frustration with the imposed ame limit for case development, saying 
that the timeframe kept them from exploring all their options. Other areas of difiiculty were reading 
legal memos onscreen, losing track of where they were in the program, and a desire for more 
infomiation to "steer^ them toward the facts of the case. 

Uqguided pairs (8/10) feh even more constrained by the time limit, citing "too much to do in 
too little time." These pairs wanted more time to "see all the possible things to do and then decide 
where to start," describing the videodisc as "such an expansive environment, it's a shame we couldn't 
take advantage of it all." One group, while lamenting the time constraint, said they Teh nished," 
then added, " but maybe this is what it's like in real life." Unguided pairs (7/10) mentioned the 
desiralrility of more instniction before getting started. One pair was quite fmstrated by their 
interaction during unannounced visits from the town attorney and repoiter. The whole point of being 
a lawyer," they lamented, " is being able to twist other people's words, not them twisting our words!" 

3. What Did You Like Most? 

Guided pairs (7/10) mentioned the process of case development using inter ac tive videodisc. 

• The process provides a look at life in the real world, is very vocational, and is a great lead-in for 

summer employment. It is a good transition between class and the real worid, providing a 
"chance to do it all." 

• There is "great benefit in having to make decisions based on missing information." 

• They felt they "got more involved in the case than they do in classroom cases" where "you get a 

packet of information handed to you." 



• In Legal Methods, 1>rief- writing happens based on facts that are given to us. This make& you 

work.*" 

• It is a good way to ''check your knowledge as you go thmugh the different phases of the case/' 

because "they all need to comt together when you file the complaint." 

• The '"uncovering process is good, and we liked maneuvering around between the parts of the 

stoiy while building the case." 

Two guided pairs mentioned the checklist: 

• I "might not have thought of doing all of it if it had not been written down." 

• [The checklist] ... "reminded me of what to do, and the instructions were good." 

Unguided pairs (9/10) mentioned design features they liked: 

• The progTLii was described as "colorful " "wondrous " "very user- friendly " and "the phone 

icon was cool." 

• The program 'Vas easy to use, especially with the mouse, because the keyboard intimidates 

me." 

• It was twice compared to a videogame in terms of being highly motivating and "having lots of 

things going on at once." 

The reality of the simulation was mentioned ofien: 

• It was "good to force us to sit at a desk and think about what to do, to bring it all together by 

making complex decisicms. It was especially good to mesh what weVe learned in class with 
what life will be like when we practice this stuff." 

• The videodisc "is more realistic than our written work in class" and it "provides experience with 

the whole surround of a legal case that has a very close resemblance to read life." 

• The personalities of the witnesses "definitely came out as we questioned them," and "it helped 

us to see how a youqg attomey could get fired." 

• It was "hard to decide, by ourselves, when we've done enough," and at times was "so much 

like real life that we didn't know when to stop talking with witnesses." 

This group also mentioned how the process affected them: 

• In Legal Methods, "we are given facts and we're learning how to be a lawyer. Here we do it 

all. This is much more challenging." 

• It "gave us an ego boost to get to act like a lawyer." 

This evidence suggests further that both the guided and unguided groups were capable of 
navigating comfortably - and in many cases thriving - in a cognitively-demanding technology 
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environment. Both groups demonstrated the ability to choose appropriate action plans» organize their 
activity in order to develq) a cohesive legal case. 

SUMMARY 

These results provide evidence that: 

1) There was variation in the pathways through Litigation Stnttegiesof first-year law students 
using two versions of orienting instmction. Guided pairs used a slightly higher number of checklist 
suggestions, began factual investigation earlier, and handled unexpected ^sitors with greater 
discretion than unguided pairs. Guided student pairs producing Superior legal complaints exhibited a 
higher level of conversation than did unguided student pairs. Unguided pairs made more efiicient use 
of their Investigative and Legal Assistants. 

2) There is a slight association between guided orienting instruction and Superior complaint 
performance. There is a stronger relationship between guided student^ success on the complaint and 
the absence of prior legal experience for this group. Student pairs producing Superior complaints 
conducted eariier investigative and later legal research than pairs producing Adequate complaints. 

3) Guided students perceived a higher level of learning than did unguided pairs when 
comparing the videodisc case development experience with traditional methods of working with the 
facts and theory of a case. These itudents fek slightly less in control of their experience, and were 
slightly less comfortable with this level of control than were unguided pairs. Finally, guided pairs 
were slightly more confident about their complaint performance. 

4) Perhaps most important, there were large differences in student attitudes about the level of 
instruction written they received prior to developing die case. While expressing slightly more 
satisfaction with the instructi<His they received, 70% of the unguided students (and 40% of the guided 
students) desired more instmctions before starting the case. It is significant that two guided students 
indicated their desire for less orienting instruction. 
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The following section of this paper examines the interrelationships among these findings^ < 
well as the implications and signiflcance of the study relative to the broader flelds of formative 
evahiation and the development of technology materials for learning. 
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DISCUSSION, LIMITATIONS, AND IMPUCATIONS 

This study was designed to learn more about law students* use of a complex videodisc 
environme A specific variable, orienting instruction, was predicted to influence students' 
pathways, performance, and perceptions of the Litigation Sttatcgics exptnence. Evidence from this 
study u used to document the importance of understanding learners' personal characteristics as well 
as their strategies for using open-ended discovery-based learning environments. This study promotes 
the va>ie of formative evaluation not only to improve the instructional design and use of such 
environments but also to inform the development of support material designed to accompany these 
environments. 

It is somewhat paradoxical to impose a mode of use within an environment based on 
exploration and discovery. There exists a tension between creating a useful orienting framework for 
students while preserving students' sense of discovery within the environment. However, the results 
of this study suggest value in providing pre-instmction to familiarize students with the organization of 
a complex computer-based environment. 

This section of the paper outlines the central findings of the study as they relate to prior 
research, and examines the significance of this research for studies in designing computer-based 
simulations; designing formative evaluation to improve computer-based products; using technology in 
legal educatim; and using orienting instniction for learning. Additionally, this chapter presents 
limitations and implications of this study for implementing cognitively-demanding videodisc 
environments with learners of varied ability, motivation, and prior experience. 

DISCUSSION OF C ENTRAL HYPOTHESES 

1. Docs Orienting Instruction Infli ^ncc th e Use of an Oocn-Ended Comouter-Based Environment? 

This hypothesis predicts differences in the way guided and unguided pairs of students use the 
Litigation Strategies program. 

o 33 
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The features of interactive videodisc technology increase the diversity of pathways students 
can take through a learning experience. As reported eailier (Hannafin, et al. J 986) this increases the 
difficulty of isolating and examining individual variables. Although variables beyond orienting 
instnictim were not isolated in this study, it is important to examine their possible influence on 
student strategies. Four factors (in additicMi to orienting instruction) may have influenced student 
pathways in this study: prior knowledge, high ability, high motivation, and working with a partner. 
Each will be discussed. 

This study found only slight differences between the case development pathways of students 
receiving two versions of orienting instruction. There exists evidence of the guided pairs' ability to 
use the orienting checklist to create a framework that helped them work through the case, supporting 
prior research (cf., Doyle & Carter, 1987). However, unguided pairs were also quite capable of 
"intuiting^ many of the strategies from the checklist without beneflt of seeing it before beginning the 
case. In previous studies, students' prior knowledge has been found to affect the need for orienting 
instiuction (Claric, 1985; Claric & VoQgel, 1985). In this study, both guided and unguided pairs had 
prior knowledge of the theory related to the litigation process. It is possible that this theoretical 
understanding prepared students for making thoughtful decisions about the practice of litigation. This 
effect corroborates previous findings from studies of advance instruction where the content is 
scmiewhat familiar to learners (cited in Melton, 1978). 

This study also conflrms the work of Levine and Loerinc ( 1 985) citing less need for pre- 
instruction for highly motivated learners. Given the enthusiasm with which students in both modes 
approached this videodisc e3q)erience, the motivation of the unguided pairs may have provided 
beneflts similar to the expected orienting instruction beneflts for guided pairs. 

The results of this stady support the flndings of Doyle (1983), which suggest that direct 
orienting instmction is most beneflcial for high ability students, forcing them to actively create their 
own understanding of new material. Questiormaire and interview data indicate that many guided pairs 
used the orienting instmction to develop a personal framework of the case, to which they referred as 
they worked. 
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The influence of working with a partner, while not examined directly in this study, is also 
important to consider. The advantages of developing the case with a partner may be similar to those 
expected from developing the case with orienting instruction. Conversation between partners 
certainly influenced many decisions - and consequent pathways *• of both the guided and unguided 
groups. It was clearly demmstrated by seveial pairs that when a particular topic is familiar to one 
partner and unfamiliar to the other, the more knowledgeable individual plays a role similar to orienting 
instruction. This could explain why students in both modes were able to work quite independently 
(i.e., as a team, without outside assistance) through the case. 

One additional factor, the element of time, may have influenced the diveisity of pairs' 
strategies in this study. As in a similar study of the influence of technology on performance (Gagnon, 
1986), it may be that the exposure time in this study (90 minutes) was not enough time to reveal 
differences in users' pathways. Or, it may be that a shorter time limit wouid exaggerate differences 
between groups, since more than half of the checklist strategies were followed during the 90-minute 
time period in this study. The influence of a specifled period of time may have limited participants' 
use of more diverse strategies that could have been further delineated. In future research, it would be 
useful to study pathways of learners with both longer and nonexisting time limits. 

Collectively, these flndings support the theory that numerous factors influence the pathways 
of students using an interactive videodisc environment such as Litigation Strategies. The next part of 
this section discusses complaint performance. 

2. Does An Imposed Mode of Use Affect C omplaint Performance? 

This hypothesis suggests that orienting instniction leads to improved understanding for 
novices using a complex learning environment (Mayer, 1979), and improved performance on a post- 
use task (Hann«ifln, et al., 1986). 

As stsiicd earlier, there was an association between guided orienting instruction and complaint 
performance. Participants in this study reported the usefulness of the pre-instmction in developing 
the case and most suggested providing even more instruction. However, the orienting instruction was 
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not found to be directly lelated to improved perfonnance on the post-use task of developing the legal 
c(miplaint. This pattern may be related to the work of Gagnon (1986), who found that (veibally) 
providing videogame players with game mles before playing had no effect on their game 
performance. In simulated experiences, especially those that students find enjoyable, the effects of 
''rules*' or Instructions" are difficuk to isolate from other variables (as mentioned earlier in this 
report). 

This study produced an equal number of Superior complaints from guided and unguided 
student pairs with prior legal experience. However, inexperienced guided pairs produced more high 
quality complaints than inexperienced unguided pairs. I attribute this finding to two possible causes. 
First, pairs with no prior real-life experience in case development may be more open to the legal 
conventions imposed by the design of the program. Second, the guided instruction may have been 
more useful to inexperienced than experienced pairs. It was expected that students recmited for this 
study would perfomi as novices in the Litigation Stiategies environment due to their limited law 
school experience. However, the strategies used by both guided and unguided student pairs indicated 
higher-level functioning in the litigation process than was expected. In future studies, it will be useful 
to determine students' prior experience and to match groups to control for previous experience. 

The effectiveness of orienting instmction may require greater interaction (between the learner 
and the orienting instmction) in order to have a more positive effect on performance. As reported 
earlier, participants in this study generally read through the checklist and then began the case, 
returning only to re-read the checklist when they became unsure of what to do next. There are at least 
three possible explanations for this: 1 ) student pairs who quickly browsed the checklist may have 
ignored most or all of the suggestions and worked on their own assumptions, 2) they may have read 
and mentally stored items they assumed would be useful, or 3) they may have become so engrossed 
in the case development experience that they actually forgot to refer back to the list. Data from this 
study provide evidence of each type of use of the checklist by guided student pairs. 

The development and application of criteria forjudging academic performance is difRcuh, 
especially in subjective areas with no "right" or "wrong" answers. In this study, the assessment of 
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complaints by legal experts was subject to each expert's philosophy of the elements constituting an 
effective legal complaint. WhUe the criteria developed by these legal experts was quite sioiilar, the 
application of those criteria varied as complaints were rated. While this is to be expected, it presents a 
problem when performance effects of a particular treatment (i.e., orienting instruction) are studied. 

In addition, it is important to consider other measures of performance within discovery-based 
environments such as Litigation Strategies. These measures involve issues and factors beyond the 
quality of the complaint. Did students enjoy their experience? Did they learn new things? These, and 
other questions are equally important to understand when considering the value of practical, 
discovery-based learning environments. The third part ofthis section discusses student perceptions 
of their own experience with Utigadm Stmtegie& 

3. Do Individuals' Perceptions of Their Videodisc Use and Per formance Vary Across Mode of Use? 

This hypothecs suggests that students may have varying beliefs about their case development 
experience, depending on an imposed mode of use (Mayer, 1979; Mandinach, 1987). 

Overall, students in both groups enjoyed the experience and were extremely motivated as they 
worked through the case. This may confirm the work of Hannafin, et al. ( 1 985), descnNng the use 
of orienting instruction to help learners focus on working in complex interactive videodisc 
environments. Althougji there is no statistical data describing the motivation and focus of the 
participants in this study, both guided and unguided groups, each receiving a form of orienting 
instruction, appeared extremely focused as they worked on the videodisc case. 

This study found that students' use of guided orienting instruction is related to slightly higgler 
levels of confidence in their woric, a slightly lower sense of personal control of their learning 
e)q>erience, and a slighdy higher estimation of the learning value of the activity. Conversely, students 
use of the unguided orienting instruction related to a slighdy higher level of enjoyment in the use of 
the videodisc, and a slightly higher impression of the disc as lifelike. 

This study corroborates the work of technology' researchers (Kurland & Pea, 1985, Newman, 
1987) studying possible positive and negative effects of providing structure in discovery-based 
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environments. There is evidence in this study that the motivation of unguided paiis» inexperienced 
pairs may have served some of the same functions as orienting instmction, and that some guided 
students desiitd less instmction than they received. 

It was suggested earlier that orienting instmction might assist participants in this study who 
felt anxious about developing the l^gal case using technology. All participants in this study had prior 
e)q>erience with computers; all but one had used a mouse as an input device. Thus» this initial concern 
appeared unnecessary. 

This study adds to the discussion about what constitutes an effective organizer for learning 
unfamiliar material (Crokill»etal.» 1988). Eariy studies supported abstract organizers (Ausubel» 
1978). Recent studies have found greater effects with concrete organizers (cited in Mandinach» 
1987). This study supports the use ofboth abstract and concrete orienting components. Guided and 
unguided pairs producing Superior complaints used more checklist suggestions than those with 
Adequate complaints. This success involved three fairly abstract suggestions (i.e.» efficient use of 
assistants^ avoid premature decisions* and use discretion with visitors), as well as the use of more 
specific checklist strategies. 

It is tempting to consider that this videodisc program may have operated easily without any 
form of pre-instruction for these students. Hannafin and Hughes ( 1986), after studying the use of 
orientir^g instruction in well-designed videodisc environments, conclude that it is less necessary to 
provide orienting instruction in high quality instructional programs. Based on the reaction to this 
program ofboth students and practicing attorneys, there is the likelihood that the environment is quite 
instmctionally-sound. However, based on a comparison of data from the pilot study and the study 
presented here, I believe that this reaction is at least in part due to the development of a clear, concise 
printed case referral memo and the printed instmctions describing the use of the technology. Thus, 
the earlier pilot study proved informative to this implementation formative evaluation of Litigation 
Stmtegic& 
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LIMITATIONS AND IMPT JCATTONS 



The goal of this research was to further our knowledge about the use of discovery-based 
videodisc experiences in education. The study was designed to explore the relationship between the 
pathways, performance, and perceptions of guided and non-guided pairs of students within a speciflc, 
complex interactive videodisc environment. 

Several limitaticms exist in this study. First, the results apply only to a small population of law 
students with beginning skills in legal education; results are not geneializable to more advanced 
students of law or to students of similar ability attending other law schools. Second, the results are 
limited to the performance of students who were asked to perform specific tasks during a 
predetermined period of time. The performance task measured only students^ ability to specify certain 
c(»nplaint criteria, versus measuring other abilities that mig^t be influenced by the environment or the 
orienting instmction (e.g., learning how to navigate within the program and explore a diversity of 
choices). Third, the results are limited by examining the perfomumce of student pairs. Interaction 
between students is described *jut is necessarily excluded as a variable to limit the scope of the study. 
Fourth, all of the data collection and most of the data analysis was conducted by one person; experts 
examining the legal complaints provided the only inter-rater data analysis in the study. It is possible 
that another evaluator may have focused on other variables (e.g., interaction between students and the 
characters onscreen) or varied in the approach used to examine and score certain aspects (e.g., 
conversation topics and scores) of students' use of this videodisc. 

With these limitati(Mis in mind, this study is informative to both legal education and the 
instructional design and development of open-ended computer-based products. Understanding the 
potential of guided instruction within such complex learning environments provides a framework for 
improving the design and implementation of these products for both students and teachers. 

The results of the study were mixed. One can hypothesize several things from these results, 
including the fact that the orienting instmction did not provide a large amount of meaningful 
information to students about the process of case development. It would seem reasonable to posit that 
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ihe orimtiqg instruction failed to have a measurable impact on perfonnance because it was not 
correlated more specifically with portions of the written legal complaint. It would also seem 
reasonable to suggest that the novelty of the technology prevented students from pursuing the case as 
rigorously as they mi^t in real life or even in a classroom setting. For example, student pairs in both 
guided and unguided groups spent more time on investigative than legal research. Interview comments 
supported this finding. It was interesting and fun to use the videodisc program to interview various 
witnesses, examine actual evidence from the case, and discuss relationships between factual 
information. Students were less enthusiastic using the program to read through and decipher various 
legal memoranda in textual form on the screen; in a real law ofllce, this task comprises a large part of 
an&lomeysjob. 

The fact that there were such positive results in one area (i.e., student perceptions), and mixed 
results in two others (i.e., pathways and performance) led us to conclude that there are advantages as 
well as disadvantages in using orienting instmction in discovery-based enviionments. Based on the 
results of this study, the question is not whether orienting instmction is effective. Rather, it is 
important to learn more about die characteristics of effective orienting instruction, and with what types 
of learners orienting instruction -^ould be used. For example, inexperienced students in this study 
benefited more from orienting instruction than experienced students. Subsequent research could 
examine tiie use of orienting instmction of varying the breadth and depth of orienting instmction with 
students of varying prior experience. 

This analysis of law students' use of an open-ended videodisc environment suggests the 
continued exploration of several important issues related to technology and learning. An important 
factor in deciding whether and how to use orienting instmction is the purpose for having students use 
the product. If the instmctional goal is to produce a rigorously investigated and analyzed product (i.e. 
the complaint), orienting instmction can provide an overview of the available ""routes'" and students 
can choose a direction. If the goal is to experience the realistic process of case development (e.g., 
experiencing surprise visits from aggressive people, going down deadends, making mistakes), a 
minimum of orienting instmction maybe more apt. 



ERIC 




Peiiiaps an even more pnctical issue in deciding how to use orienting instniction involves the 
geneial availability of the videodisc equipment, as well as the amount of time allotted to the excfdse by 
the instnictor. If the equipment and attitude of an instnictor permit unlimited time to "discover^ the 
relationship between the facts and the theory of the casCy less structure might be imposed at the outset. 
However, if students need to get quickly into - and out of- Hit exercise, an orienting framework 
could provide a structure within which to work more expediently. 

The design ofLitigatioa StrategicstepTcsents a departure from the traditional use of computer- 
based environments for learning. Its use within the law school environment, while brxiadening the 
scope of the traditional legal educationiu curriculum, may be enhanced by the availability of varying 
types of orienting instruction for learners varying background. This broadened view of using 
technology represents the hope that technology may allow teachers to help leamers at many levels lO 
achieve new types cf goals in school curriculums goals that were previously reserved for real life 
and professional experience on the job. 
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TABLE 4: MEANOU^rVOtSATIONSOORES 
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TABLES: MEANSTB»OFlN!TUT10NOFINVE5TiaATIVE AND LEGAL RESEARCH 
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TABLE 9: PRXXt LEGAL EXPBUENCE 
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TABLE 10: MEAhS AND STANDARD DEVUTIOhS FOR QUC^nONNAlRE FTEMS 
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2. How«aqrordiffic«tci*«itioMd«nuiiddiccMtr 1.95 1.7S 

(l^M)r...5-diffiadi) <0.89> <0.«S) 

3. Howad«|««torlMd«qwMiMrtlfc«ia«c«oiout 2.50 2.20 

(l-iad«qat«...5«iMd«qwt«) (1.00) (.83) 

4. HowoQafidMiar»yo«oryo«rooiiiplaiKcrii«iaT 3.15 2.S0 

(l«flOooaf...5-eonpl«ttco^ (0.81) (1.15) 

5. Ho«m«cfaooaBraldidyMka««o¥vdiii<xp«tiMoer 3.40 3.<5 

(1— occBgol...5»cinwpl<r«coflirol) (0.99) <1.04) 

6. Howoofifoctiibltw««yo«witfcdiiilcvdafoMiroir 3.00 2.55 

(laiwycooif ...5a>v«x«aooB0 (1-26) (1-32) 

7. HowBocfadidyoaiUbt^vortd^wkhapmiMct 4.25 4.10 

(l«aoc««U...5>«loc) (0.79) (1.21) 

4. Howm«didtdyoacajoyMuvLitigKioiiScratcgi«sr 4.00 4.30 

(l-ooc «c «U ... 5«a tat) (086) (0.92) 

9. How muck ^diliitcKpcricaoe resemble real life? 3.15 3.45 

(l-nocacaU...5«v«rymach) (1.14) (1.10) 

10. How moch did you learn abooc iavcsigatii^ a case? 3.d5 .00 

(Isvciy little. ..5«aloc) (0.87) (0.92) 
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TABLE U: ADEQUACY OP INSTRUCTIOMSI 





OUIDED 


UNOUIDED 


COMBINED 


MORElNffrRUCnC 


• (40«) 


14(70%) 


22 (53«> 


hP CHANGE 


\0(SO%) 


600%) 


16 (40<f> 


LESSWSTRUCnOl 


2(J0«) 1 




2(>0«) 




(20) 


ao) 


(40) 



t - bMttI oa opinloM of .-Ift ««plt (40 IftdivldMU) 
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